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Response to Arguments 

1 . Applicant's arguments filed 19 and 29 have been fully considered but they are not 
persuasive. 

Applicant argues (see Remarks, page 7) that applicant's invention is 
characterized by agent software that monitors closed caption signal, and that, "the 
agent automatically looks for specific words or phrases in one format (e.g. text), that 
trigger a speech recognizer to watch for that word in another format (e.g. audio)" and 
therefore, "is different than scanning text records for a user selected search term". 
While the examiner recognizes differences between applicant's invention and Herz, 
the claim language are generally recited very broadly and therefore reads on the 
system of Herz. Amended limitation of "determining a search parameter" fails to 
overcome the rejection and therefore fails to place the application in condition for 
allowance. 

2. Examiner further notes that applicant did not traverse the examiner's assertion of 
official notice on claims 24 and 31 , stating that, it was well known in the art at the 
time of the invention "to display still images on video" or examiner's assertion of the 
official notice on claim 33, 34 and 39, stating that, "text to speech converters were 
well known in the art at the time of the invention to convert search parameter into 
audio data when a user enters the search parameter in text format". Accordingly, 
these common knowledge statements are taken to be admitted prior art. 

For the reasons stated above, the rejection is maintained. 

Claim Rejections - 35 USC § 102 
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3. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form 
the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the. United States and was published under Article 21(2) 
of such treaty in the English language. 

4. Claims 19-23, 25-26, 29-30, 32, 37, and 40-41, are rejected under 35 U.S.C. 102(e) 

as being anticipated by Herz (US Pat. 5,914,719). 

With regards to claims 19, Herz discloses a method of searching in a 
multimedia signal (abstract, column 2, lines 1-4), comprising: 

Receiving a search parameter (search term) corresponding to a first data 
format component (text data) of the multimedia signal, the multimedia signal 
including the first data format component (text data such as CC) and a second data 
format component (video/audio); see column 5, lines 23-24. 

Searching the multimedia signal to identify an occurrence of the search 
parameter in the first data format component of a multimedia signal; see column 5, 
lines 24-25. The system further determines the occurrence of a search parameter 
by searching for the occurrence of the search parameter in the first data format 
component of the multimedia signal. 
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Determining a portion of the second data format component of the multimedia 
signal that corresponds to the identified occurrence of the search parameter in the 
first data format component of the multimedia signal; see column 1, lines 54-55. 

Synchronizing a first segment and a second segment of the multimedia signal 
(aligning text, audio, video using identifiers), wherein. the first segment includes the 
occurrence of the search parameter in the first data format component of the 
multimedia signal and the second segment includes the portion of the second data 
format component of the multimedia signal that corresponds to the occurrence of the 
search parameter in the first data format component. See column 4, lines 40-44, 
lines 52-54, and column 5, lines 40-45. 

With regards to claim 20, and 22, the first data format is a closed caption of 
the multimedia signal (see column 2, lines 45-47) and the second data format is an 
audio/video component of the multimedia signal (see column 5, lines 40-41). 

With regards to claim 21 and 23, the second segment of the multimedia signal 
is a second of the audio/video component that begins and ends within a 
predetermined period of time before and after the occurrence of the search 
parameter in the closed caption component (see column 5, lines 40-46). 

With regards to claims 25, the first data format is a closed caption of the 
multimedia signal (see column 2, lines 45-47) and the second data format is an 
audio/video component of the multimedia signal (see column 5, lines 40-41). 
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With regards to claims 26, and 40, the first and second data formats are 
selected from the group consisting of: text data, audio data and video data. See 
column 2, lines 45-47 and column 5, lines 40-41 . 

With regards to claim 29, Herz discloses a method of searching in a 
multimedia signal (abstract, column 2, lines 1-4), comprising: 

Searching the multimedia signal to identify an occurrence of the search 
parameter in the first data format component of a multimedia signal; see column 5, 
lines 24-25. The system accordingly analyzes a first data format of the multimedia 
signal in order to determine/identify the occurrence of a search parameter in the first 
data format component of the multimedia signal. 

Determining a portion of the second data format component of the multimedia 
signal that corresponds to the identified occurrence of the search parameter in the 
first data format component of the multimedia signal; see column 1, lines 54-55. 

Synchronizing a first segment and a second segment of the multimedia signal 
(aligning text, audio, video using identifiers), wherein the first segment includes the 
- occurrence of the search parameter in the first data format component of the 
multimedia signal and the second segment includes the portion of the second data 
format component of the multimedia signal that corresponds to the occurrence of the 
search parameter in the first data format component. See column 4, lines 40-44, 
lines 52-54, and column 5, lines 40-45. 
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With regards to claim 30, the second data format is a video component (see 
column 5, lines 40-41) and the system displays the segment of the video component 
to the user (see column 5, line 45-47). 

With regards to claim 32, the segment of the video component is a video clip 
of predetermined length substantially corresponds to an occurrence of the search 
parameter in the first data component (see column 5, lines 40-46). 

With regards to claim 37, Herz discloses a method of processing multimedia 
signal, comprising the steps of: 

Receiving a search parameter (see column 5, lines 23-24) of a first data 
format (see abstract); 

Processing the multimedia signal to determine an occurrence of the search 
parameter in a first data component of the multimedia signal (see column 5, lines 24- 
25), wherein the multimedia signal has at least the first component (text data such 
as CC) and a second component (AN data), the first component being in a first data 
format and the second component being in a second data format. The system 
further determines the occurrence of a search parameter by searching for the 
occurrence of the search parameter in the first data format component of the 
multimedia signal. 

Determining a portion of the second component of multimedia signal that 
corresponds to the occurrence of the search parameter in the first component. See 
column 1, lines 54-55. 
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With regards to claim 41 , Herz further discloses the step of synchronizing the 
first segment and the second segment of the multimedia signal (aligning text, audio, 
video using identifiers), wherein the first segment includes the occurrence of the 
search parameter in the first component of the multimedia signal and the second 
segment includes the occurrence of the second component of the multimedia signal 
that corresponds to the occurrence of the search parameter in the first component. 
See column 4, lines 40-44, lines 52-54, and column 5, lines 40-45. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 1 02 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

6. Claims 24, and 31, are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Herz (US Pat. 5,914,719). 

With regards to claim 24, and 31, Herz is silent that the second component is 
a still image from the video component that is present substantially at the occurrence 
of the search parameter in the closed caption component. 

Examiner takes official notice that it is well known in the art to display still 
images on video. Herz further discloses that the identifier for locating the search 
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query within the video and text data maybe a time stamp data. See Herz: column 4, 
lines 13-15. 

It would have been obvious to one of ordinary skill in the art at the time of the 
invention to modify the system of Herz by allowing the user to submit query for 
retrieval of still images present in video signals, so that the user can locate the 
desired segment of the video. The modified system uses identifiers with time 
stamps for locating the still image corresponding to the user query, thereby retrieving 
second component substantially at the occurrence of the search parameter in the 
closed caption component. 

7. Claims 27-28, 33-36, and 38-39 are rejected under 35 U.S.C. 103(a) as being 

J 

unpatentable over Herz (US Pat. 5,914,719) in view of Brodsky (US Pat. 5,809,471). 

With regards to claims 27, 35, and 38, Herz is silent on the step of receiving a 
search parameter in a third data format (such as audio) different from the first data 
format, and converting the search parameter from the third data format to the first 
data format. 

In a similar field of endeavor, Brodsky discloses the step of receiving a search 
parameter in a third format (such as voice) and using voice type dictation tools that 
converts the voice data to text data. See column 4, line 64-column 5, line 3, and 
column 6, lines 24-26. By receiving and recognizing the search parameter in a third 
format such as voice, the search parameter is 'determined in order to subsequently 
convert the voice data to text data. 
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It would have been obvious to one of ordinary skill in the art to modify the 
system in view of Brodsky, by allowing to enter search parameter in a third data 
format such as voice and using speech to text converter for converting the voice 
data to text (i.e. first data format) in order to compare the search parameter with the 
closed caption data. The motivation is to allow user an intuitive method of entering 
search queries such as voice commands. 

With regards to claim 28, the modified system comprises the step of search 
parameters in a voice (i.e. third data format) format and therefore the third data 
format is selected from the group comprising audio data. 

With regards to claim 33 and 39, Herz discloses the step of enabling the user 
to enter a text search parameter (see abstract). Herz however does not disclose 
that the first data component is an audio component, wherein the search parameter 
is converted to an audio data format prior to analyzing the multimedia signal. 

In a similar field of endeavor, Brodsky discloses the method of searching for 
spoken words or audio information (see column 4, lines 18-22) in a signal. Brodsky 
further discloses that user search request for a keyword maybe made via a direct or 
a remote selection from a displayed dictionary menu (i.e. France), wherein the 
requested word is matched against the words in dictionary database. Accordingly 
Brodsky discloses a method of comparing a displayed keyword (i.e. text information) 
with stored keywords (i.e. "spoken words" or audio information) of a first data 
component. Examiner additionally takes official notice that text to speech converters 
were well known in the art at the time of the invention to convert search parameter 
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into audio data in order to compare search parameter with the audio component of 
the multimedia signal, when a user enters the search parameter in text format. 

It would have been obvious to one of ordinary skill in the art at the time of 
the invention to modify the system of Herz by allowing the user to select a keyword 
of text format and search for the occurrence of the keyword in an audio signal by 
converting the text keyword into audio data, thereby providing means for searching 
in a multimedia signal that does not have any text data components. 

With regards to claim 34, the step of converting text search parameter into an 
audio data is performed using a text to speech converter in the modified system. 

With regards to claim 36, the step of converting speech search parameter 
into text data is performed using a speech to text converter (voice type dictation 
solution) in the modified system. 

Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory action 
is mailed, and any extension fee pursuant to 37 CFR 1 .136(a) will be calculated from 
the mailing date of the advisory action. In no event, however, will the statutory 
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period for reply expire later than SIX MONTHS from the mailing date of this final 
action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Usha Raman whose telephone number is (571) 272- 
7380. The examiner can normally be reached on Mon-Fri: 9am-6pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Christopher Kelley can be reached on (571) 272-7331. The 
fax phone number for the organization where this application or proceeding is 
assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR 
only. For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571- 
272-1000. 




